Towards Efficient Probabilistic Temporal Planning

Many real-world planning problems involve a combination of both time and uncertainty (Bresina et al. 2002). For instance, Aberdeen et al. (2004) investigate military operations planning problems that feature concurrent durative actions, probabilistic timed effects, resource consumption, and competing cost measures. It is the potential for such practical applications that motivates this research. Probabilistic temporal planning combines concurrent durative actions with probabilistic effects. This unification of the disparate fields of probabilistic and temporal planning is relatively immature, and presents new challenges in efficiently managing an increased level of expressiveness. Some of our techniques for solving probabilistic temporal planning problems could be applied beyond the context they were developed in, and may prove useful in efficiently solving the simpler subproblems. The most general probabilistic temporal planning framework considered in the literature is that of Younes and Simmons (2004). It is expressive enough to model generalised semi-Markov decision processes (GSMDPs), which allow for exogenous events, concurrency, continuous-time, and general delay distributions. This expressiveness comes at a cost: the solution methods proposed in (Younes & Simmons 2004) lack convergence guarantees and significantly depart from the traditional algorithms for both probabilistic and temporal planning. Concurrent Markov decision processes (CoMDPs) are a much less general model that simply allows instantaneous probabilistic actions to execute concurrently (Guestrin, Koller, & Parr 2001; Mausam &Weld 2004). Aberdeen et al. (2004) and Mausam and Weld (2005) have extended this model by assigning actions a fixed numeric duration. They solved the resulting probabilistic temporal planning problem by adapting existing MDP algorithms, and have devised heuristics to help manage the exponential blowup of the search space. The ultimate goal of this research is to produce planners that are expressive enough to support: concurrent durative actions, probabilistic effects, metric resources, and cost functions; while being efficient enough to solve interestingsized (real-world) problems. We currently have two separate avenues of research with the aim of achieving this goal. The first approach is to combine a forward-chaining search with effective heuristics. We have developed a probabilistic temporal planner called Prottle using this approach (Little, Aberdeen, & Thiébaux 2005). Prottle uses a (deterministic) trialbased search algorithm with a heuristic that is based on an extension of the planning graph data structure. Another approach to planning is the Graphplan framework (Blum & Furst 1997). While Prottle makes use of the planning graph—a data structure that originates from this framework—it does not use the framework’s other key features; in particular, Prottle does not use a backward search. The Graphplan framework has previously been successfully applied to temporal planning (concurrent durative actions) (Smith &Weld 1999), but had not been successfully applied to probabilistic planning (actions with probabilistic effects) in its entirety. Extensions of this framework for probabilistic planning had been developed (Blum & Langford 1999), but either dispense with the techniques that enable concurrency to be efficiently managed, or are unable to produce optimal contingency plans. As a way of investigating approaches to compressing the search space for probabilistic temporal planning, our other avenue of research has the goal of implementing a probabilistic temporal planning in the Graphplan framework. As the issues relating to probabilistic planning had not been adequately solved, and as a way of managing the complexity, we started by developing a (concurrent) probabilistic planner (Little & Thiébaux 2006). Paragraph, the resulting planner, is competitive with the state of the art, producing acyclic or cyclic plans that optionally exploit a problem’s potential for concurrency. We are confident that this approach can be extended to the probabilistic temporal context. This paper gives a brief overview of both Prottle and Paragraph, and concludes with remarks about our future research intentions. For more detailed descriptions and experimental results, please refer to the respective papers (Little, Aberdeen, & Thiébaux 2005; Little & Thiébaux 2006).